Image pickup apparatus and processing method for result of image pickup

ABSTRACT

The present invention is applied to an image pickup apparatus for which, for example, a CMOS solid-state image pickup element is used. One screen image is divided into a plurality of blocks, and a motion is detected for each of the blocks to control the exposure time of the block.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.14/159,616, filed on Jan. 21, 2014, which is a continuation of U.S.application Ser. No. 13/690,999, filed on Nov. 30, 2012 (U.S. Pat. No.8,687,069), which is a divisional of U.S. application Ser. No.11/794,996, filed on Oct. 15, 2007, (U.S. Pat. No. 8,345,108), whichapplication is a national phase entry under 35 U.S.C. §371 ofInternational Application No. PCT/JP2005/022603 filed Dec. 2, 2005,which claims priority from Japanese Application No. P2005-006178 filedJan. 13, 2005, all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Technical Field

This invention relates to an image pickup apparatus and a processingmethod for a result of image pickup and can be applied, for example, toan image pickup apparatus for which a CMOS solid-state image pickupelement is used. The present invention makes it possible to furtherenhance the picture quality in comparison with conventionally availablepicture quality in regard to control of the exposure time by means of anelectronic shutter by dividing a screen image into a plurality ofblocks, performing motion detection and controlling the exposure timefor each of the blocks.

2. Background Art

Conventionally, an image pickup apparatus is configured such that thecharge accumulation time of an image pickup element is controlled by anelectronic shutter to control the exposure time so that, also in a casewhen an image of an image pickup object which is moving quickly ispicked up, an image pickup result having no image shake can be acquired.Regarding such control by an electronic shutter as just described, amethod is proposed, for example, in Japanese Patent Laid-open No. Hei6-165047 and Japanese Patent Laid-open No. Hei 6-261256 wherein a motionof an image pickup object is detected by a system controller and theexposure time is controlled based on the motion detection resulttogether with execution of gain control of AGC (Auto Gain Control) andcontrol by ALC (Auto Light Control).

However, conventional control of the exposure time by an electronicshutter is performed such that, where the exposure time is variedequally for all of pixels which form one screen image to vary theexposure time based on a motion of a portion of the screen image, theexposure time is reduced also regarding a stationary portion of thescreen image such as a background or the like. Therefore, in such a caseas just described, there is a problem that the S/N ratio at such astationary portion as just described deteriorates and the picturequality deteriorates as much.

Conversely, there is another case wherein any motion cannot be detectedin regard to a small object. In this case, since it is difficult tocontrol the exposure time appropriately, an image of the small object ispicked up in a blurred state. Therefore, also in this instance, there isa problem that the picture quality deteriorates.

DISCLOSURE OF INVENTION

The present invention has been made taking the foregoing intoconsideration and proposes an image pickup apparatus and a processingmethod for an image pickup result by which the picture quality can beenhanced significantly when compared with conventional picture qualitywith regard to control of the exposure time by an electronic shutter.

In order to solve the subject described above, the present invention isapplied to an image pickup apparatus including: image pickup meanshaving an image pickup surface formed thereon from pixels arranged in amatrix for outputting a result of image pickup of an optical imageformed on the image pickup surface; motion detection means for detectinga motion amount in the result of the image pickup for each of blocksinto which the image pickup surface is divided in a horizontal directionand/or a vertical direction; control means for controlling exposure timeof the image pickup means for each of the blocks based on a result ofthe detection by the motion detection means such that the exposure timedecreases as the motion amount increases; and signal level correctionmeans for correcting and outputting, for each of the blocks, a signallevel of the result of the image pickup which varies in accordance withcontrol of the exposure time.

According to the configuration of the present invention, where it isapplied to an image pickup apparatus including: image pickup meanshaving an image pickup surface formed thereon from pixels arranged in amatrix for outputting a result of image pickup of an optical imageformed on the image pickup surface; motion detection means for detectinga motion amount in the result of the image pickup for each of blocksinto which the image pickup surface is divided in a horizontal directionand/or a vertical direction; and control means for controlling exposuretime of the image pickup means for each of the blocks based on a resultof the detection by the motion detection means such that the exposuretime decreases as the motion amount increases, the exposure time can becontrolled in response to a motion for each of the blocks. Consequently,regarding a stationary background, the exposure time can be set long toprevent degradation of the S/N ratio, but regarding a portion whichinvolves some motion, the exposure time can be set short to prevent ablurred state. Further, by detecting a motion for each of the blocks, amotion can be detected also from a small object, and as a result, ablurred state can be prevented also with regard to such a small objectas described above. Consequently, regarding control of the exposure timeby an electronic shutter, the picture quality is enhanced significantlyin comparison with conventionally available picture quality. Further,since the signal level correction means for correcting and outputting,for each of the blocks, a signal level of the result of the image pickupwhich varies in accordance with control of the exposure time isprovided, a variation of the signal level between the blocks whichvaries in accordance with control of the exposure time for each of theblocks in this manner can be prevented.

Further, the present invention is applied to a processing method for aresult of image pickup, including: a motion detection step of detectinga motion amount in a result of image pickup by image pickup means foreach of blocks into which an image pickup surface of the image pickupmeans is divided in a horizontal direction and/or a vertical direction;a control step of controlling exposure time of the image pickup meansfor each of the blocks based on a result of the detection at the motiondetection step; and a signal level correction step of correcting andoutputting, for each of the blocks, a signal level of the result of theimage pickup which varies in accordance with control of the exposuretime.

Consequently, according to the configuration of the present invention, aprocessing method for a result of image pickup by which the picturequality can be enhanced significantly in comparison with conventionallyavailable picture quality regarding control of the exposure time by anelectronic shutter can be provided.

According to the present invention, the picture quality can be enhancedsignificantly in comparison with conventionally available picturequality regarding control of the exposure time by an electronic shutter.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a general configuration of anintegrated circuit according to an embodiment of the present invention.

FIG. 2 is a sectional view of a sensor chip of the integrated circuit inFIG. 1.

FIG. 3 is a side elevational view illustrating a lamination structure ofthe sensor chip and a logic chip.

FIG. 4 is a block diagram showing a detailed configuration of theintegrated circuit of FIG. 1.

FIG. 5 is a timing chart illustrating operation of a pixel block of FIG.4.

FIG. 6 is a timing chart illustrating control of the exposure time ofpixels.

FIG. 7 is a view illustrating a concept of control of the exposure timeof blocks.

FIG. 8 is a block diagram showing an image pickup apparatus according toa second embodiment of the present invention.

FIG. 9 is a block diagram showing an image pickup apparatus according toa third embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following, embodiments of the present invention are described indetail suitably with reference to the drawings.

(1) Configuration of the First Embodiment

FIG. 1 is a block diagram showing an integrated circuit applied to animage pickup apparatus according to a first embodiment of the presentinvention. The image pickup apparatus according to the presentembodiment forms a desired optical image on an image pickup surface ofthe integrated circuit 1 by automatic iris control and automaticfocusing control by a controller for which a lens not shown is used andoutputs a result of the image pickup of the optical image.

Here, the integrated circuit 1 is an integrated circuit including animage pickup element and is formed by packaging a laminated object of asensor chip 2 and a logic chip 3.

Here, the sensor chip 2 is a semiconductor chip of an image pickupelement which outputs a result of image pickup in accordance with an XYaddress method, and, in the present embodiment, a CMOS solid-state imagepickup element is applied as the image pickup element. The sensor chip 2includes an image pickup section 4 and a control section 5 forcontrolling operation of the image pickup section 4.

Here, the image pickup section 4 has an image pickup surface formed byarranging pixels in a matrix and outputs a result of image pickup of anoptical image formed on the image pickup surface. In the image pickupsection 4, the pixels arranged in a matrix are divided equally in ahorizontal direction and a vertical direction, and the image pickupsurface is divided into a plurality of pixel blocks 6 thereby. Aperipheral circuit relating to pixels which belong to each of the pixelblocks 6 is provided individually for the pixel block 6, and the pixelblocks 6 are individually formed such that a result of image pickupthereof is outputted. Therefore, the sensor chip 2 outputs the resultsof image pickup by the plural pixel blocks 6 simultaneously andparallelly.

As shown partly in section in FIG. 2, in the sensor chip 2, an elementlayer 12 is formed from a silicon (Si) layer having a thickness ofapproximately 10 to 20 [μm] and a light reception element 13 is formedon the element layer 12. Further, a silicon dioxide (SiO₂) film 14, alight blocking film 15, a silicon nitride film (SiN) 16, a color filter17, and a micro lens 18 are successively laminated as an upper layerwith respect to a portion corresponding to the light reception element13, and a pixel 19 is formed thereby. Further, the peripheral circuitsof the pixel blocks 6, the control section 5 and a wiring layer forwiring the peripheral circuits and the control section 5 are formed as alower layer with respect to the element layer 12, and a substratesupporting member 21 for supporting the entirety is provided on thelower layer side of the wiring layer 20. Consequently, in the sensorchip 2, the wiring layer 20 is disposed on the side opposite to that ofthe image pickup surface and the peripheral circuits, control section 5and so forth are provided. Thus, the sensor chip 2 is configured suchthat decrease of the numerical aperture can be avoided effectively alsoin a case wherein peripheral circuit and so forth are providedindividually for the pixel blocks 6. Further, the sensor chip 2 isformed such that various constraints where the wiring layer 20 is formedon the image pickup surface side are cancelled and the degree of freedomin wiring can be increased significantly.

It is to be noted that the sensor chip 2 is formed in such a mannerthat, since the wiring layer 20 is formed on the side opposite to thatof the image pickup surface, a semiconductor substrate having a smallthickness is processed from the wiring layer 20 side to form the lightreception element 13 and circuit elements of the peripheral circuits andthen the wiring layer 20 and substrate supporting member 21 aresuccessively formed on the semiconductor substrate, and thereafter, theelement layer 12 is completed by reversing and polishing thesemiconductor substrate using a CMP and the light blocking film 15,silicon nitride film (SiN) 16, color filter 17 and micro lens 18 areformed successively.

As shown in FIG. 3, in the sensor chip 2, the logic chip 3 is allocatedto the substrate supporting member 21, and the sensor chip 2 iselectrically connected to and supported on the logic chip 3 by fineprojection electrodes formed on the wiring layer 20 side and fineprojection electrodes 32 formed on the logic chip 3. The fine projectionelectrodes 32 are hereinafter referred to as micro bumps.

Here, the logic chip 3 is an integrated circuit including a signalprocessing circuit for processing a result of image pickup from thesensor chip 2, and, in the present embodiment, the signal processingcircuit is formed from a pulse production section 41 for controlling theexposure time of the image pickup section 4 provided on the sensor chip2 and a control section 42 for outputting various timing signals to thepulse production section 41.

Here, in the pulse production section 41, pulse production blocks 43individually corresponding to the pixel blocks 6 of the sensor chip 2are provided. The pulse production blocks 43 are individually connectedto the pixel blocks 6 corresponding to the micro bumps 31 and 32 so thatthe exposure time of the corresponding pixel blocks 6 is controlled.Consequently, the integrated circuit 1 is formed such that results ofimage pickup from the plural pixel blocks 6 can be outputtedsimultaneously and parallelly and the exposure time periods of the pixelblocks can be individually controlled by the pulse production blocks 43.

FIG. 4 is a block diagram showing the pixel block 6 and pulse productionblock 43. Here, in the pixel block 6, pixels 19A and 19B placedsuccessively in a horizontal direction are connected to a reset controlcircuit 52 through a common gate line. Further, the pixels 19A and 19Bplaced successively in a vertical direction are connected to each otherthrough a common signal line and are connected to a horizontal drivingcontrol circuit 53. The pixels 19A and 19B perform an opto-electricconversion process for incoming light to accumulate electric charge andthen discharge the accumulated electric charge in accordance with resetpulses RST1 and RST2 outputted from the reset control circuit 52.Further, the pixels 19A and 19B convert the accumulated electric chargeinto a voltage in accordance with readout pulses ROUT1 and ROUT2outputted from the reset control circuit 52 and output the resultingvoltage to the signal line. Consequently, in the pixel block 6, bycontrol of the reset control circuit 52, the electric chargeaccumulation time is varied to vary the exposure time and a result ofthe image pickup is outputted in a unit of a line to the horizontaldriving control circuit 53.

The reset control circuit 52 resets operation in accordance with avertical synchronizing signal VD outputted from the control section 5 asshown in (A) of FIG. 5 and (A) of FIG. 6. Then, the reset controlcircuit 52 causes the reset pulse RST1 to rise at a timing of a risingedge of a shutter pulse SHT (refer to (B) of FIG. 5 and (B) of FIG. 6)outputted from a pulse production block 43 (refer to (C1) of FIGS. 5 and(C1) of FIG. 6, and causes a readout pulse ROUT1 to rise at a timing ofa falling edge of the shutter pulse SHT (refer to (D1) of FIGS. 5 and(D1) of FIG. 6).

The reset control circuit 52 outputs the reset pulse RST1 and thereadout pulse ROUT1 generated with reference to the shutter pulse SHT tothe pixel 19A on the top line. Further, the reset control circuit 52successively delays the reset pulse RST1 and the readout pulse ROUT1 bydelay time Δt corresponding to the number of lines which form the pixelblocks 6 to successively generate reset pulses RST2, . . . , and RSTn,and readout pulses ROUT2, . . . , and ROUTn relating to the pixels 19B,. . . of the remaining lines (refer to (C2) and (D2) of FIGS. 5, and(C2), (D2), (Cn) and (Dn) of FIG. 6) and outputs the reset pulse RST2and the readout pulse ROUT2, . . . to the pixels 19B, . . . of the otherlines.

Consequently, the reset control circuit 52 sets such that the periods T1and T2 from a rising edge to a falling edge of the shutter pulse SHT areset as exposure time periods for the pixels 19A and 19B. Further, thereset control circuit 52 controls operation of the pixels 19A and 19Bsuch that the accumulated electric charge (refer to (E1) and (E2) ofFIG. 5) accumulated in the pixels 19A and 19B is outputted as a resultof pickup image to the signal lines in a unit of a line at a timing ofan end of the exposure time (refer to (F1) and (F2) of FIG. 5).

Further, the reset control circuit 52 outputs a gain control signal Gfor an amplification circuit 54 in accordance with the exposure timebased on the shutter pulse SHT, and then corrects the signal level of aresult of image pickup which varies in accordance with control of theexposure time based on the shutter pulse SHT.

Consequently, results of image pickup of the pixels 19A, 19B, . . .placed successively in a vertical direction are successively outputtedto the signal lines by time division (refer to (E) of FIG. 6). Thehorizontal driving control circuit 53 time division multiplexes andoutputs the results of image pickup outputted from the pixels 19A and19B to the signal lines in a unit of a line in such a manner asdescribed above (refer to (F) of FIG. 6).

The amplification circuit 54 varies the gain in accordance with the gaincontrol signal G outputted from the reset control circuit 52, andamplifies and outputs the result of image pickup outputted from thehorizontal driving control circuit 53. Consequently, the integratedcircuit 1 corrects the signal level of the result of image pickup whichvaries in accordance with control of the exposure time and outputs theresult of image pickup of the corrected signal level.

An analog/digital conversion circuit (ADC) 55 performs an analog/digitalconversion process for the output signal of the amplification circuit 54and outputs image data OUT according to a result of the process to thecorresponding pulse production block 43 of the logic chip 3.Consequently, in the present embodiment, the peripheral circuits of thepixels 19A and 19B belonging to the pixel blocks 6 are formed from thereset control circuit 52, horizontal driving control circuit 53,amplification circuit 54 and analog/digital conversion circuit 55.

Consequently, the integrated circuit 1 is configured such that the imagepickup results OUT by the pixels 19A and 19B are outputtedsimultaneously and parallelly from the pixel blocks 6 in such a mannerthat the image pickup results OUT are outputted from the pixel blocks inaccordance with an order of raster scanning and the exposure timeperiods by the electronic shutter are varied individually by the pixelblocks 6 under the control of the corresponding pulse production block43, and variations of the signal levels in the image pickup results OUTby variation of the speed of the exposure time are correctedindividually in the pixel blocks 6.

The pulse production block 43 detects a motion amount of the result ofthe image pickup outputted from the corresponding pixel blocks 6 bymeans of the motion amount detection circuit 61 and varies and outputsthe shutter pulse SHT in accordance with a result of the motion amountdetection.

In particular, in the motion amount detection circuit 61, an averagingcircuit 62 averages results of image pickup outputted from thecorresponding pixel blocks 6 in a unit of a frame. In particular, theaveraging circuit 62 cumulatively adds the results of image pickupoutputted from the corresponding pixel blocks 6 by means of a summingcircuit 63 and a delay circuit 64, and outputs a result of the summationto a frame difference calculation circuit 66 for each of frames andinitializes the sum value under the control of a switch circuit 65provided at an output stage of the summing circuit 63. Consequently, themotion amount detection circuit 61 cumulatively adds the results ofimage pickup in a unit of a frame and then calculates an average valueof the results of image pickup from the pixel blocks 6.

The frame difference calculation circuit 66 delays the output value ofthe averaging circuit 62 by a period of time of one frame by means ofthe delay circuit (1V delay) 67. Further, the frame differencecalculation circuit 66 subtracts the output value of the delay circuitfrom the output value of the averaging circuit 62 by means of asubtraction circuit 68 to calculate an interframe difference value ΔS.Consequently, the motion amount detection circuit 61 calculates themotion amount approximately based on the interframe difference value ΔS.

The shutter pulse production circuit 69 produces the shutter pulse SHTin accordance with the result of motion amount detection obtained insuch a manner as described above by the motion amount detection circuit61. In particular, the shutter pulse production circuit 69 produces theshutter pulse SHT such that, if the interframe difference value ΔSoutputted from the motion amount detection circuit 61 increases, thenthe period of time of a rising edge of the shutter pulse SHT is reducedas much. Consequently, the shutter pulse SHT is produced such that, ifthe motion amount increases, then the exposure time reduces as much.

In the process, the shutter pulse production circuit 69 delays, if theinterframe difference value ΔS outputted from the motion amountdetection circuit 61 increases, the timing of a rising edge of theshutter pulse SHT as much, and advances the timing of a falling edge ofthe shutter pulse SHT by the amount by which the timing of the risingedge is delayed. Consequently, the shutter pulse production circuit 69performs setting such that, even if the shutter speed is varied, thetiming of the center of the exposure time in the pixel blocks 6 does notvary so that the phase of the shutter pulse SHT does not vary by a greatamount among the pixel blocks 6. It is to be noted that this makes thetime from a rising edge to a next rising edge of the shutter pulse SHTone-half the vertical synchronization period at the shortest.Accordingly, the reset control circuit 52 is configured such that thereset pulse RST1 and the readout pulse ROUT1 are produced from theshutter pulse SHT and are successively delayed to produce the resetpulses RST2, . . . and the readout pulses ROUT2, . . . for the lines,and then the reset pulses RST1, RST2, . . . and the readout pulsesROUT1, ROUT2, . . . are outputted individually within periods of thefront half and the rear half of the vertical synchronization period.

Consequently, the integrated circuit 1 sets the shutter speed shorteronly at locations at which a motion is detected from image pickupresults OUT(F1), OUT(F2) and OUT(F3) successively obtained by the sensorchip 2 in accordance with the interframe difference values ΔS of theresults of image pickup OUT(F1), OUT(F2) and OUT(F3) as seen in FIG. 7.Consequently, degradation of the S/N ratio at a stationary portion isavoided effectively thereby to prevent a blurred state of a moving imagepickup object. Further, also in a case wherein the image pickup objectwhich exhibits a motion is a small object, the shutter speed of theportion relating to the small object is set shorter thereby to prevent ablurred state with certainty. It is to be noted that, in FIG. 7, theexposure time of the pixel blocks with reference to a case wherein theexposure time is not set shorter at all is indicated by a fraction.

Then, the integrated circuit 1 accumulates the image pickup results OUToutputted simultaneously and parallelly from the pixel blocks 6 in sucha manner as described above into an image memory provided in the sensorchip 2 and multiplexes the image pickup results OUT in accordance withan order of raster scanning and then outputs the multiplexed results ina single series.

Consequently, in the present embodiment, the pixels 19A and 19B, resetcontrol circuit 52 and horizontal driving control circuit 53 form imagepickup means for outputting a result of image pickup of an optical imageformed on an image pickup surface, and the motion amount detectioncircuit 61 forms motion detection means for detecting a motion amount ofthe result of the image pickup for each of blocks formed by dividing theimage pickup surface in a horizontal direction and/or a verticaldirection. Further, the shutter pulse production circuit 69 formscontrol means for controlling the exposure time of the image pickupmeans for each of the blocks, based on a result of the detection by themotion detection means such that the exposure time reduces as the motionamount of the block increases, and the reset control circuit 52 and theamplification circuit 54 form signal level correction means forcorrecting and outputting, for each of the blocks, the signal level ofthe result of the image pickup which varies by control of the exposuretime.

(2) Operation of the First Embodiment

In the image pickup apparatus (FIG. 1) having the configurationdescribed above, an optical image is formed on the image pickup surfaceof the integrated circuit 1 through the lens and a result of the imagepickup of the optical image is acquired by and outputted from theintegrated circuit 1. In the process of the integrated circuit 1, in theintegrated circuit 1 (FIGS. 2 and 4), the image pickup surface is formedby disposing the pixels 19A and 19B of the light reception element 13 ina matrix, and image pickup results OUT are obtained from the pixelblocks 6 formed by equally dividing the image pickup surface in ahorizontal direction and a vertical direction.

In particular, in the pixel blocks 6, the readout pulses ROUT1, ROUT2, .. . whose timings are successively shifted on the lines are outputtedfrom the reset control circuit 52, and image pickup results out1, out2,. . . by the pixels 19A and 19B are inputted to the horizontal drivingcontrol circuit 53 in a unit of a line in accordance with the readoutpulses ROUT1, ROUT2, and then time division multiplexed by thehorizontal driving control circuit 53. Further, after the signal levelof the image pickup results are corrected by the succeedingamplification circuit 54, the image pickup results are converted intoand outputted as digital signals by and from the analog/digitalconversion circuit 55. Consequently, an image pickup result OUT isobtained for each of the pixel blocks 6 formed by dividing the imagepickup surface, and the image pickup results OUT are outputtedsimultaneously and parallelly.

Further, the image pickup results OUT of the pixel blocks 6 are inputtedindividually to the corresponding pulse production blocks 43, by whichthe motion amounts ΔS of the pixel blocks 6 are determined. Then, basedon the motion amounts ΔS, the electric charge accumulation time of thecorresponding pixel blocks 6 is controlled such that the exposure timereduces as the motion amount of the pixel block increases.

Consequently, in the present image pickup apparatus, the exposure timecan be controlled in response to a motion for each of blocks formed bydividing one screen image, and, regarding a stationary background, theexposure time can be set long to prevent degradation of the S/N ratio,but regarding a portion which involves some motion, the exposure timecan be set short to prevent a blurred state. Further, by detecting amotion for each of blocks, a motion can be detected also from a smallobject, and as a result, a blurred state can be prevented also withregard to such a small object as described above. Consequently,regarding control of the exposure time by an electronic shutter, thepicture quality is enhanced significantly in comparison withconventionally available picture quality.

Further, in the present image pickup apparatus, the exposure time iscontrolled in such a manner as described above to increase the gain ofthe amplification circuit 54 to correct the signal levels of the imagepickup results OUT by an amount corresponding to an amount of reductionof the exposure time. Consequently, the exposure time can be controlledfor each of the blocks in such a manner as described above to preventthe variation of the signal levels among the blocks, and degradation ofthe picture quality can be prevented thereby.

By controlling the exposure time for each of the pixel blocks in thismanner, in the present image pickup apparatus, the image pickup resultsOUT are acquired for the individual pixel blocks 6 and outputtedsimultaneously and parallelly, and the motion amounts ΔS of the pixelblocks 6 are detected to control the exposure time by the pulseproduction blocks 43 individually corresponding to the pixel blocks 6.Consequently, in the present image pickup apparatus, the exposure timecan be controlled for each of the blocks independently of each other bysimultaneous and parallel processing for the plural blocks, and, as aresult, the picture quality is enhanced by the easy process incomparison with conventionally available picture quality.

Particularly, in the present image pickup apparatus, the configurationthat the sensor chip 2 and the logic chip 3 are connected and laminatedby the micro bumps and 32 such that the image pickup results OUT areacquired by the sensor chip 2 and the exposure time is controlled by thelogic chip 3 to control the exposure time for each of the blocks bysimultaneous and parallel processing of the plural blocks independentlyof each other in such a manner as described above can be applied to sucha lamination structure of the sensor chip 2 and the logic chip 3 asdescribed above to simplify the general configuration.

Further, in the sensor chip 2, by disposing the peripheral circuits suchas the reset control circuit 52, horizontal driving control circuit 53and so forth on the side opposite to the image pickup surface andlaminating the peripheral circuits on the opposite side surface togetherwith the logic chip 3, decrease of the numerical aperture of the pixels19A and 19B can be avoided effectively, and further, crosstalk betweenthe pixels 19A and 19B adjacent each other and so forth can bedecreased. Further, the occupation area of the pixels 19A and 19B on theimage pickup surface can be sufficiently secured, and, as a result,refinement of the pixels can be facilitated thereby to facilitatefabrication of the image pickup apparatus. Further, the degree offreedom in connection to the succeeding logic chip 3 can be enhancedsignificantly, and the degree of freedom in design can be enhanced asmuch.

By performing motion detection to control the exposure time in thismanner, in the pulse production block 43, the averaging circuit 62cumulatively sums the image pickup results OUT in a unit of a frame todetermine an average value of the image pickup results OUT in eachframe. Then, the succeeding frame difference calculation circuit 66determines interframe differences of the average values to determinemotion amounts ΔS. Further, the shutter pulse production circuit 69produces shutter pulses SHT from the motion amounts ΔS, and the resetcontrol circuit 52 produces reset pulses RST1, RST2, . . . from theshutter pulses SHT and varies the timings of the readout pulses ROUT1,ROUT2, . . . to vary the exposure times. Consequently, in the presentimage pickup apparatus, a motion amount ΔS is calculated by a simple andeasy process of determining an interframe difference, and a process forexposure control is executed. Consequently, the exposure control processcan be executed for each block by a simple and easy configuration.

(3) Effects of the First Embodiment

According to the configuration described above, by dividing one screenimage into a plurality of blocks and performing motion detection foreach of the blocks to control the exposure time for each block, thepicture quality can be enhanced significantly in comparison withconventionally available picture quality with regard to control of theexposure time by an electronic shutter.

Further, at this time, by outputting image pickup results of the blockssimultaneously and parallelly and processing the image pickup results ofthe blocks to detect the motion amount for each of the blocks, theexposure time can be controlled by simultaneous and parallel processeswhich are independent of each other for each block. Consequently, thepicture quality can be enhanced significantly in comparison withconventionally available picture quality regarding the control of theexposure time by an electronic shutter by simple and easy processes.

Further, by cumulatively summing image pickup results for each of theblocks in a unit of a frame to calculate average values and calculatinginterframe difference values of the average values to detect a motionamount for each of the blocks, the motion amount can be detected by asimple and easy configuration, and the general configuration can besimplified as much.

Further, by forming at least pixels of image pickup means, peripheralcircuits to the pixels and a configuration of motion detection means onsemiconductor chips different from one another and connecting themthrough micro bumps, the entire apparatus can be formed as an integratedcircuit to simplify the configuration thereby to prevent decrease of thenumerical aperture of the pixels and further assure the degree offreedom in design sufficiently.

(4) Second Embodiment

As shown in FIG. 8, an image pickup apparatus 71 according to thepresent embodiment includes encoding means 72 and recording means 73 inaddition to the configuration of the integrated circuit 1 describedhereinabove in connection with the first embodiment. The encoding means72 and the recording means 73 perform an encoding process for imagepickup results OUT and record a result of the encoding process on arecording medium. Therefore, in the image pickup apparatus 71 accordingto the present embodiment, an image pickup system 74 is formed from theintegrated circuit described hereinabove in connection with the firstembodiment, and image pickup results OUT outputted from the image pickupsystem 74 are multiplexed and then converted into and inputted as aluminance signal and a color difference signal to the encoding means 72.Then, resulting data of an encoding process by the encoding means 72 arerecorded on the recording medium by the recording means 73.

Here, the encoding means 72 is encoding means relating to datacompression in which a motion vector is used, and performs an encodingprocess of the image pickup results OUT using, for example, thetechnique of H.264.

On the other hand, the image pickup system 74 sets the number of pixelsto form a pixel block 6 so as to correspond to a macro block to be usedfor detection of a motion vector by the encoding means 72. It is to benoted that, in this instance, a pixel block may be set so as tocorrespond to a macro block of one of the luminance signal and the colordifference signal. Further, according to H.264, motion detection may beperformed from a plurality of kinds of macro blocks of different sizessuch that a pixel block is set so as to correspond to a macro block ofone of the plural kinds of macro blocks.

Further, the image pickup system 74 sets the pixel blocks 6 in thismanner and outputs motion amounts ΔS detected by the pixel blocks 6.

The encoding means 72 determines the motion amounts ΔS with apredetermined threshold value to determine presence or absence of amotion. Then, if it is determined as a result of the determination thata motion is involved, then a notion vector is detected by an ordinaryprocess and an encoding process of the image pickup result OUT isperformed. On the other hand, if it is determined as a result of thedetermination that no motion is involved, then detection of a motionvector is suspended and the motion vector is set to the value zero toperform an encoding process for the image pickup result OUT.

Consequently, in the present embodiment, a motion detection result usedin control of the exposure time is utilized in an encoding process suchthat motion detection is performed for each block to control theexposure time, and the burden of the encoding process is moderated asmuch.

According to the present embodiment, by dividing image pickup resultsinto blocks with a size corresponding to a detection unit of a motionvector in an encoding process and performing motion detection for eachof the blocks to control the exposure time of each block and thenutilizing results of the motion detection in the encoding process, thepicture quality can be enhanced significantly in comparison withconventionally available picture quality regarding the control of theexposure time by an electronic shutter. Consequently, the burdeninvolved in the encoding process can be reduced.

(5) Third Embodiment

As seen in FIG. 9 in contrast to FIG. 8, an image pickup apparatus 81according to the present embodiment detects a motion amount by detectionof a motion vector in place of detection of a motion amount by aninterframe difference by the motion detection circuit 61 (refer to FIG.4) of the image pickup system 74 described hereinabove in connectionwith the second embodiment. In particular, a motion vector V isdetected, and this motion vector is converted into an absolute value todetect the length of the motion vector V, and then the magnitude of themotion is detected from the length of the motion vector V. It is to benoted, to the detection of the motion vector V, various detectionmethods such as a block matching method and a slope method can beapplied.

Further, encoding means 82 suspends, in detection of a motion vector Vregarding a block of a size corresponding to a pixel block 6, adetection process of a motion vector, and executes an encoding processusing the motion vector V detected by the motion detection circuit 61.It is to be noted that the image pickup apparatus 81 is configuredsimilarly to the image pickup apparatus 71 described hereinabove inconnection with the second embodiment except such configurations whichrelate to a motion vector V as described above.

According to the present embodiment, similar effects to those of thesecond embodiment can be achieved also where a motion is detected usinga motion vector to control the exposure time for each block.

(6) Other Embodiments

It is to be noted that, while, in the embodiments described above, thetimings of both of a reset pulse and readout pulse are varied to controlthe exposure time, according to the present invention, the control ofthe exposure time is not limited to this, but the exposure time may becontrolled by variation of the timing only of a reset pulse.

Further, while, in the embodiments described above, an image pickupresult is outputted and processed for each pixel block, according to thepresent invention, outputting of an image pickup result is not limitedto this, but image pickup results for one screen image may be outputtedcollectively and then classified and processed for each image block tocontrol the exposure time for each of the pixel blocks. By this, evenwhere an image pickup element of a conventional configuration whichoutputs image pickup results in accordance with an order of rasterscanning is used in regard to outputting of image pickup results, thepicture quality can be enhanced significantly in comparison withconventionally available picture quality regarding the control of theexposure time by an electronic shutter similarly as in the embodimentsdescribed hereinabove

Further, while, in the embodiments described hereinabove, a sensor chipis formed from a CMOS solid-state image pickup element, the presentinvention is not limited to this but can be applied widely to a casewherein a sensor chip is formed from various solid-state image pickupelements according to the XY address system and another case wherein asensor chip is formed from a CCD solid-state image pickup element.

Further, while, in the embodiments described hereinabove, the imagepickup surface is divided in a horizontal direction and a verticaldirection to form pixel blocks, according to the present invention,formation of pixel blocks is not limited to this, but the image pickupsurface may be divided only in one of a horizontal direction and avertical direction to form pixel blocks.

Further, while, in the embodiments described hereinabove, peripheralcircuits to the pixels are provided on the rear surface side of thesensor chip 2 and laminated together with a logic chip, the presentinvention is not limited to this but can be applied widely also to acase wherein the peripheral circuits to the pixels are provided on thefront surface side of the sensor chip 2, another case wherein a logicchip is formed as a separate member and so forth.

INDUSTRIAL APPLICABILITY

The present invention can be applied to an image pickup apparatus inwhich, for example, a CMOS solid-state image pickup element is used.

1. (canceled)
 2. An imaging device comprising: a first semiconductorsubstrate including an image pickup section having a first pixel blockand a second pixel block; a second semiconductor substrate stacked onthe first semiconductor substrate and including a first pulse productionblock and a second pulse production block configured to control exposuretime of the image pickup section; and a connection portion that connectsthe first semiconductor substrate and the second semiconductorsubstrate, wherein the first pulse production block is electricallyconnected to the first pixel block, wherein the second pulse productionblock is electrically connected to the second pixel block, and whereinthe second semiconductor substrate includes: a first motion amountdetection circuit that processes an output of the first pixel block; anda second motion amount detection circuit that processes an output of thesecond pixel block in parallel with the first motion amount detectioncircuit.
 3. The imaging device according to claim 2, wherein the firstsemiconductor substrate further comprises a micro lens and a colorfilter, and wherein a first side of the color filter is disposedadjacent to the micro lens.
 4. The imaging device according to claim 3,wherein the first semiconductor substrate further comprises a nitridefilm, a light shielding member, and an oxide film, a second side of thecolor filter is disposed adjacent to a first side of the nitride film,and a first side of the light shielding member is disposed adjacent to asecond side of the nitride film and a second side of the light shieldingmember is disposed adjacent to a first side of the oxide film.
 5. Theimaging device according to claim 4, wherein the first semiconductorsubstrate comprises a light reception element, and the micro lens isdisposed over the light reception element.
 6. The imaging deviceaccording to claim 5, wherein a center line of the micro lens is shiftedto a center line of the light reception element in a cross-sectionalview.
 7. The imaging device according to claim 5, wherein the firstsemiconductor substrate further comprises a wiring layer, a first sideof the light reception element is disposed adjacent to a second side ofthe oxide film, and a second side of the light reception element isdisposed adjacent to a first side of the wiring layer.
 8. The imagingdevice according to claim 7, wherein the wiring layer has at least a4-layered wiring structure.
 9. The imaging device according to claim 7,wherein a second side of the wiring layer is adjacent to the secondsemiconductor substrate.
 10. An imaging device comprising: a firstsemiconductor substrate including an image pickup section having a firstblock including a first plurality of pixel circuits and a second blockincluding a second plurality of pixel circuits; a second semiconductorsubstrate stacked on the first semiconductor substrate and including apulse production circuit configured to output a driving signal to one ofthe first plurality of pixel circuits and the second plurality of pixelcircuits; a connection portion that connects the first semiconductorsubstrate and the second semiconductor substrate, wherein the pulseproduction circuit is electrically connected to one of the firstplurality of pixel circuits and the second plurality of pixel circuits,wherein the second semiconductor substrate includes: a first signalprocessing circuit that processes an output of the first plurality ofpixel circuits; and a second signal processing circuit that processes anoutput of the second plurality of pixel circuits in parallel with thefirst signal processing circuit.
 11. The imaging device according toclaim 10, wherein the first semiconductor substrate further comprises amicro lens and a color filter, and wherein a first side of the colorfilter is disposed adjacent to the micro lens.
 12. The imaging deviceaccording to claim 11, wherein the first semiconductor substrate furthercomprises a nitride film, a light shielding member, and an oxide film, asecond side of the color filter is disposed adjacent to a first side ofthe nitride film, and a first side of the light shielding member isdisposed adjacent to a second side of the nitride film and a second sideof the light shielding member is disposed adjacent to a first side ofthe oxide film.
 13. The imaging device according to claim 12, whereinthe first semiconductor substrate comprises a light reception element,and the micro lens is disposed over the light reception element.
 14. Theimaging device according to claim 13, wherein a center line of the microlens is shifted to a center line of the light reception element in across-sectional view.
 15. The imaging device according to claim 13,wherein the first semiconductor substrate further comprises a wiringlayer, a first side of the light reception element is disposed adjacentto a second side of the oxide film, and a second side of the lightreception element is disposed adjacent to a first side of the wiringlayer.
 16. The imaging device according to claim 15, wherein the wiringlayer has at least a 4-layered wiring structure.
 17. The imaging deviceaccording to claim 15, wherein a second side of the wiring layer isadjacent to the second semiconductor substrate.
 18. The imaging deviceaccording to claim 17, wherein the connection portion connects thewiring layer and the second semiconductor substrate.
 19. The imagingdevice according to claim 10, wherein the first signal processingcircuit is adjacent to the second signal processing circuit.
 20. Theimaging device according to claim 10, wherein each of the first signalprocessing circuit and the second signal processing circuit isconfigured to detect a motion amount for each of the blocks.
 21. Animaging device comprising: a first substrate including: a first blockincluding a first plurality of pixels, wherein at least one of the firstplurality of pixels is configured to output a first pixel signal; and asecond block adjacent to the first block, the second block including asecond plurality of pixels, wherein at least one of the second pluralitypixels is configured to output a second pixel signal; a second substratelaminated to the first substrate, including: a first signal processingcircuit that processes the first pixel signal; and a second signalprocessing circuit that processes the second pixel signal in parallelwith the first signal processing circuit; and a connection portion thatconnects the first substrate and the second substrate.
 22. The imagingdevice according to claim 21, wherein the first substrate furthercomprises a micro lens and a color filter, wherein a first side of thecolor filter is disposed adjacent to the micro lens.
 23. The imagingdevice according to claim 22, wherein the first substrate furthercomprises a nitride film, a light shielding member, an oxide film, asecond side of the color filter is disposed adjacent to a first side ofthe nitride film, and a first side of the light shielding member isdisposed adjacent to a second side of the nitride film and a second sideof the light shielding member is disposed adjacent to a first side ofthe oxide film.
 24. The imaging device according to claim 23, whereinthe first substrate comprises a light reception element, and the microlens is disposed over the light reception element.
 25. The imagingdevice according to claim 24, wherein a center line of the micro lens isshifted to a center line of the light reception element in across-sectional view.
 26. The imaging device according to claim 24,wherein the first substrate further comprises a wiring layer, a firstside of the light reception element is disposed adjacent to a secondside of the oxide film, and a second side of the light reception elementis disposed adjacent to a first side of the wiring layer.
 27. Theimaging device according to claim 26, wherein the wiring layer has atleast a 4-layered wiring structure.
 28. The imaging device according toclaim 26, wherein a second side of the wiring layer is adjacent to thesecond semiconductor substrate.
 29. The imaging device according toclaim 28, wherein the connection portion connects the wiring layer andthe second semiconductor substrate.
 30. The imaging device according toclaim 21, wherein the first signal processing circuit is adjacent to thesecond signal processing circuit.
 31. The imaging device according toclaim 21, wherein each of the first signal processing circuit and thesecond signal processing circuit is configured to detect a motion amountfor each of the blocks.